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Abstract 

Background, Goal and Scope. For the life cycle assessment (LCA) 
tool to provide maximum benefit for decision makers, the un¬ 
certainty of its results should be reported. Several methods for 
assessing uncertainty have been developed, but despite recent 
efforts, there remains disagreement about their merits. 
Objectives. The objectives of the study were to review several 
assessment methods for estimating numerical and qualitative 
uncertainty of impact scores and recommend an appropriate 
uncertainty assessment scheme. The methods review has been 
conducted on the basis of an LCA case study regarding the com¬ 
parison of the use of either brown or black coals in Australian 
electricity generation. 

Results and Discussion. Each assessment method indicated 
greater uncertainty in the impact scores calculated for black coal 
use than for brown coal use. Due to overlap of the uncertainty 
ranges in calculated impact scores neither of the coals could be 
regarded environmentally preferred. 

Conclusions. Both qualitative and quantitative methods were 
found to provide useful information about the uncertainty of 
calculated impact scores for the case study. Methods that com¬ 
bine qualitative and quantitative uncertainty provided no addi¬ 
tional benefits, and obscured much of the information gained 
from using qualitative methods. 

Recommendation and Outlook. It is recommended that LCA 
results should include separate numerical (using Monte-Carlo 
simulation) and qualitative uncertainty assessments. When the 
ranges of calculated impact scores for compared options over¬ 
lap, the normalised difference method is recommended. 


Keywords: Australian electricity generation; coal; data quality; 
data uncertainty; life cycle assessment (LCA) 


Introduction 

One of the stated purposes of the life cycle assessment (LCA) 
tool is to provide decision makers with information about 
the environmental effects of choices. Despite the ISO stand¬ 
ards' recommendation that data quality parameters be re¬ 
ported [1], data quality assessment is not current practice in 
LCA studies [2,3,4]. Therefore, in practice, decision makers 
must make a judgement about the accuracy of the LCA. 


However, without some systematic assessment of data qual¬ 
ity there is no adequate basis for that judgement. 

One method of expressing the practitioner's confidence in 
the results, and increasing the probability of trust, is to ac¬ 
knowledge the range of possible (or likely) outcomes. The 
environmental impact scores calculated using average, maxi¬ 
mum, and minimum data values might result in entirely dif¬ 
ferent relativities for alternatives. Thus, the range of impact 
assessment scores obtainable from the LCA should be re¬ 
ported, and possible conclusions discussed. 

The aim of this paper is to review and compare several meth¬ 
ods to determine, which method, or combination of meth¬ 
ods, provides the best option to assess the uncertainty of the 
LCA results. This is achieved by undertaking an assessment 
of data quality with regard to a case study on environmen¬ 
tal impacts in electricity generation from Australian black 
and brown coals, which is part of a wider study on electric¬ 
ity generation in Australia [5]. 

1 Review of Uncertainty Methods 
1.1 Introduction 

Traditional methods for determining the range of a result 
(known as error analysis), used in other types of assessment, 
have proven inadequate for LCA, as they cannot include so 
many different types of uncertainty [6,7]. Thus, significant 
effort has occurred to develop a generalised method that 
can be applied in LCA. 

Guidance on data quality analysis for LCAs can be obtained 
from three main sources. Firstly, the International Organi¬ 
zation for Standardization (ISO) 14040s series of standards 
state that data quality requirements should be documented 
during the development of the study's Goal and Scope Defi¬ 
nition [1], including: age, geography, and technology infor¬ 
mation; the precision, completeness, and representativeness 
of the data; the consistency and reproducibility of the LCA 
model; the data sources and their representativeness; and 
the uncertainty of the information [1]. If there is a need for 
an accurate and detailed study [8] three types of analyses 
are recommended: gravity analysis (e.g. Pareto), which de¬ 
termines data with greatest contribution; uncertainty analy¬ 
sis, which determines range of possible results based on in- 


215 





Australian Electricity Generation 


LCA Case Studies 


Table 1 : Data quality pedigree matrix by Weidema and Wesnaes [9] 



Indicator Score 

Indicators 

1 

2 

3 

4 

5 


Independent of the study in which the data are applied: 


Reliability of the 
source 

Verified data based 
on measurements 

Verified data partly 
based on 

assumptions or non- 
verified data based 
on measurements 

Non-verified data 
partly based on 
assumptions 

Qualified estimate (e.g. 
by an industrial expert) 

Non-qualified estimate or 
unknown origin 

Completeness 

Representative data 
from a sufficient 
sample of sites over 
an adequate period to 
even out normal 
fluctuations 

Representative data 
from a smaller 
number of sites but 
for adequate periods 

Representative data 
from an adequate 
number of sites but 
for shorter periods 

Representative data 
from a smaller number 
of sites and shorter 
periods, or incomplete 
data from an adequate 
number of sites and 
periods 

Representativeness 
unknown or incomplete 
data from a smaller 
number of sites and/or 
from shorter periods 


Dependent on the goal and scope of the study: 


Temporal 

correlation 

Less than 3 years of 
difference to year of 
study 

Less than 6 years of 
difference to year of 
study 

Less than 10 years 
of difference to year 
of study 

Less than 15 years of 
difference to year of 
study 

Age unknown or more than 

15 years of difference to 
year of study 

Geographical 

correlation 

Data from area under 
study 

Average data from 
larger area in which 
the area under study 
is included 

Data from area with 
similar production 
conditions 

Data from area with 
slightly similar 
production conditions 

Data from an unknown 
area or with very different 
production conditions 

Technological 

correlation 

Data from enterprises, 
processes and 
materials under study 

Data from processes 
and materials under 
study but from 
different enterprises 

Data from 
processes and 
materials under 
study but from 
different technology 

Data on related 
processes and 
materials but from same 
technology 

Unknown technology or 
data on related processes 
or materials but from 
different technology 


dividual data uncertainties; and sensitivity analysis, which 
determines the effect that changes in data and methods have 
on the results [1,8]. In this paper, only the uncertainty analysis 
is considered, as this is an area of great complexity in LCA. 
Secondly, the Centre of Environmental Science (CML) Guide 
[7] recommends that the ISO requirements be followed, with 
the optional use of a 'Pedigree Matrix' and/or one of the 
proposed uncertainty frameworks (e.g. [6], or [3]). Finally, 
the Society of Environmental Toxicology and Chemistry 
(SETAC) European working group on 'Data Availability and 
Data Quality' [4] provide both a framework for uncertainty, 
and possible methods to calculate data uncertainty. The 
framework classifies data uncertainty into 'data inaccuracy' 
and 'lack of specific data' (i.e. data gaps and unrepresenta¬ 
tive data). Data gaps are to be filled using either input-out¬ 
put modelling, data from similar products, or mass balances. 
Unrepresentative data uncertainty is estimated by applying 
uncertainty factors, obtained through further analysis of ma¬ 
terial inputs and outputs, to account for temporal, geographi¬ 
cal and technological differences, using the pedigree matrix of 
Weidema (Table 1). Total uncertainty is obtained using the 
Monte Carlo simulation method (see Section 1.2). Empirical 
justification of the uncertainty factors and data ranges used is 
also necessary to ensure the relevance of the conclusions. This 
is a substantial task for even a simple LCA study. 

A review of methods indicated that three types existed: a 
quantitative assessment considering only numerical uncer¬ 
tainties in input data (see Section 1.2); a qualitative assess¬ 
ment using indicators to consider a range of input data as¬ 
pects (see Section 1.3); and a quantitative assessment, where 
the qualitative input data aspects are converted into quanti¬ 
tative uncertainty (see Section 1.4). 


1.2 Numerical uncertainty 

Quantitative assessment methods use input data variability 
information to obtain a cumulative uncertainty value for an 
output, using uncertainty propagation. Proposed analytical 
methods for uncertainty propagation include uncertainty 
propagation, interval methods, fuzzy logic simulations, and 
Bayesian statistics [3]. However, stochastic models, such as 
Monte Carlo [3,10] or its modified form Latin Hypercube 
[3] simulation, are most commonly used, as they are able to 
include any number of different probability distributions for 
the individual data points [3]. These stochastic methods ran¬ 
domly vary all individual data values, using their individual 
uncertainty distributions. For example, if a data point has a 
mean at 100, with a distribution such that the probability 
that it is 100 is 50%, while the probability that it is 95 is 
12%, then for 100 iterations, a result for the entire system 
will be calculated using a value of 100 for this data point in 
approximately 50 iterations, and a value of 95 in approxi¬ 
mately 12 iterations. As the calculation involves many such 
data points, all varying independently from each other, the 
result calculated for each iteration will also vary. The col¬ 
lected results from all these iterations form a new profile, 
which represents the uncertainty of the result. 

Where possible, actual ranges and probability distributions of 
data collected are used; however, when data is limited, 'rules 
of thumb' [11,12] may be used to produce likely ranges. In 
this case, the mean value, and estimated minimum and maxi¬ 
mum values, or a 95% confidence interval, can be used to 
define a probability profile for each data source. Many prob¬ 
ability profiles may be chosen, however the most common 
are: triangular, uniform, normal, and log-normal (Fig. 1). 
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Fig, 1 : Common probability distributions (a = minimum, b = average, c = 
maximum) 


Despite the seeming completeness of such methods, they do 
suffer a number of limitations. Firstly, they consider only 
one aspect of data quality, variability from the true mean, 
ignoring aspects such as the data's fitness for purpose, and 
most importantly, how accurately the system modelled in 
the LCA matches reality [6,13]. Secondly, combining the un¬ 
certainties of the different types of data used at different points 
in the LCA model has proven difficult [6,14], e.g. data uncer¬ 
tainty and uncertainty due to allocation between products. 
Thirdly, the uncertainty of the LCA model itself is unknown 
[7], and cannot be tested against real systems, as the outputs 
have no defined spatial or temporal characteristics [6]. Lastly, 
the final uncertainty value can only be calculated with confi¬ 
dence for a specific impact category [6]. While relative con¬ 
tributions of specific impacts to total environmental impact 
can be postulated, for example by reference to expert opinion, 
such relativities are necessarily subjective and their uncertainty 
cannot be quantified on a scientific basis. 


1.3 Qualitative uncertainty 

To measure the qualitative uncertainty in an LCA model 
each of the attributes of the system that add to it are judged 
independently and given a score; a lower score in most cases 
represents higher quality. The scores may then be interpreted 
on three levels [6]: 

• Data level (individual data sources) 

Example indicators: see Table 1 

• Process level (physical or notional subsections of the system) 
Example indicators: 

- Completeness (all flows considered in process) and 

- Applicability of assumptions for allocation and aggregation 

• System level 
Example indicators: 

- Completeness (all processes considered) 

- Applicability of assumptions for aggregation and impact cat¬ 
egory choices 

This method is often called the 'Pedigree Matrix' method [15]. 

Many groups of indicators have been proposed to describe 
an LCA system. For example, Weidema and Wesnaes [9] 
proposed the indicators shown in Table 1. Each of these 
indicators is data level only, while others (i.e. [13], or [16]) 
list indicators at many levels. Some declare that these indi¬ 
cators may not be aggregated because the scores do not rep¬ 
resent an amount of quality (i.e. [9], [17], or [19]), and thus 
the aggregated scores could not be used for comparisons. 
However, other researchers (i.e. [16] and Lindeijer et al. 1 ) 


suggest otherwise, as the combined scores are only designed 
to indicate, rather than measure. 

In one method, proposed by Wrisberg [16], quality indica¬ 
tors are aggregated to obtain the total system quality, using 
equal weights for all environmental flows. Consider for ex¬ 
ample, the case where an environmental impact result is 
developed from two material flows, with data quality scores 
of (3, 2, 4, 1, 1) and (1, 1, 2, 3, 4), for the five indicators 
respectively, and contribute 33% and 67% of the environ¬ 
mental impact result respectively. The final indicator scores 
will equal the sum of the indicators divided by two, or (2, 
1.5, 3, 2,2.5). 

In another method, proposed by Rousseaux et al. [13], the 
quality performance of each data point is compared to some 
decided target quality goal score (e.g. 2 out of 5). To deter¬ 
mine the performance of a particular process or system, the 
percentage of data points that pass this criterion are com¬ 
pared (called 'Acceptability'). For example, if a hypotheti¬ 
cal process A has three data points with quality indicator 
scores of (1,2,1, 3, 4) for point B, (2, 3,2, 4,1) for point C, 
and (3, 3, 3, 3, 3) for point D, and the chosen quality goal 
score is 2, then for the first indicator: point B's (1) and C's 
(2) score passes, but point D’s score (3) fails. Thus the 'ac¬ 
ceptability' of process A's first indicator is 66% 2 . If the prac¬ 
titioner is attempting to reduce qualitative uncertainty (i.e. 
indicator scores) then another parameter, the ratio of the 
quality score variance to the average quality score is calcu¬ 
lated. Variance here is the sum of each points difference from 
the mean divided by the number of points. Using the above 
example, the average of process A's first indicator is 2, its 
variance is 2/3, and thus its variability ratio is 33 % 3 . If this 
ratio is high, then the quality scores have a wide spread, and 
may thus include some very 'good' (1) and 'bad' (5) scores. In 
this case, the total quality may be improved by reducing the 
number of 'bad' scores. When the ratio is low, then the quality 
scores are in a narrow range and greater numbers of data 
points must be changed to improve the total data quality score. 


1.4 Methods for combining numerical and qualitative 
uncertainty 

Numerical and qualitative uncertainty results, obtained from 
the methods shown in Sections 1.2 and 1.3, together may be 
used to indicate uncertainty in impact results. Alternatively, 
an individual data point's qualitative uncertainty can be con¬ 
verted into a quantitative uncertainty profile (called 'additional 
uncertainty') and added to the data point's numerical uncer¬ 
tainty profile (see Section 1.2). This combined profile can then 
be used to generate a quantitative uncertainty profile for the 
impact results, adding process and system level qualitative 
uncertainties at the appropriate calculation points during a 
stochastic calculation method (see Section 1.2). 

Three different methods have been proposed for the conver¬ 
sion of qualitative uncertainty into additional uncertainty: 


1 From: Lindeijer, E., Berg, N.W. van den and Huppes G. (1997), Proce¬ 
dure for Data Quality Assessment, Report for Rioned (in Dutch), Sep¬ 
tember (discussed in [6]). 


2 For the remainder of the indicators the 'acceptability's' are 33%, 66%, 
0%, and 33%. 

3 For the remainder of the indicators the 'reliability's' are 8%, 33%, 7%, and 
58%. 
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Table 2: Calculation table to obtain the data quality indicator of Kennedy et at. [10] 


% of Attainable Data Quality (x) 

Aggregated Data Quality 
Indicator (1) 

Shape Parameters 

I Extents (% variation from mean) 

a 

P 

A 

IH pm 

100 

5 

5 

5 

-10 

+10 

87.5 <x< 100 

4.5 

4 

4 

-15 

+15 

75 < x < 87.5 

4 

3 

3 

-20 

+20 

62.5 < x < 75 

3.5 

2 

2 

-25 

+25 

50 < x < 62.5 

3 

1 

1 

-30 

+30 

37.5 < x < 50 

2.5 

1 

1 

-35 

+35 

25 < x < 37.5 

2 

1 

1 

-40 

+40 

12.5 < x < 25 

1.5 

1 

1 

—45 

+45 

0<x< 12.5 

1 

1 

1 

-50 

+50 


Beta probability functions [10]; additional estimated uncer¬ 
tainty ranges (twice the coefficient of variation 4 ) for each 
type of data elements [18]; and obtaining estimates of vari¬ 
ation due to the problem indicated 5 [9]. The first method 
(of Kennedy et al. [10]) integrates a chosen list of indicators 
through a parameter, x, known as the percent of attainable 
data quality (Equation 1). It does not recommend a pre¬ 
scribed set of qualitative indicators, but recommends that 
they be developed from previous experience of using the 
method. To illustrate the Kennedy et al. method, if a data 
point has a value of 100, with quality scores of (2,2,3,2,1) 
for five indicators, then the sum of quality scores is 10, the 
sum of maximum quality scores is 25 (maximum score for 
each indicator is 5), and the sum of minimum quality scores is 
5 (minimum score for each indicator is 1). Thus in this exam¬ 
ple x equals (10-5)/(25-5) times 100 = 25%. The data quality 
indicator (I) is obtained from Table 2 (example I is 2). This 
indicator (I) is then used to create one or more Beta probabil¬ 
ity distributions, which are modified normal distributions, 
described using two shape parameters (a and P) and its ex¬ 
tents (A and B). These parameters are estimated from I using 
expert knowledge, or from Table 2 (in the example: a = 1; P = 
1; A = -0.4 times the value of the data point or -40; and B = 
0.4 times the value of the data point or 40). 

| Z Quality Scores — Z Minimum Quality Scores j 

\ Z Maximum Quality Scores - Z Minimum Quality Scores J ^ ^ 

The second method (Meier [18]) is designed for data, proc¬ 
ess, and system level indicators, including indicators for as¬ 
sumptions, valuation, and impact category considerations. 
Its conversion factors are developed from previous experi¬ 
ence in using the method. For example, the conversions in 
Table 3 were developed for the geographical correlation in¬ 
dicator. In this case, additional data uncertainty is calcu¬ 
lated for the chosen indicators using the additional uncer¬ 
tainties shown in Table 3, and is characterised by a normal 
distribution with standard deviation of 1/4 of this range. 
For the example data above, (2,2,3,2,1), the additional un- 


4 The coefficient of variation is equal to the standard deviation divided by 
the mean value. 

5 For example, to update energy consumption data taken from an older 
process, the increased efficiency of the new process could be used. In 
the general case, this requires additional research into each data point, 
and thus considerable additional resources may be necessary. 


Table 3: Example additional uncertainty calculation table forthe geographi¬ 
cal correlation indicator of the Meier method [6] 


Quality Score (1) 

Additional Uncertainty Range (±%) v '" 

1 

5 

2 

10 

3 

20 

4 

30 

5 

50 


certainty will equal the sum of that for each indicator: 
±(10+10+20+10+5)%, or ± 55%. Using the mean value of 
our example, 100, then the variation is ± 55, and the stand¬ 
ard deviation is (45 - (—55))/4 or 27.5. 

Despite the mathematical nature of these methods, the results 
are still subjective, reliant on expert judgement to determine 
the variation caused by each indicator score [17]. As a single 
range for each impact category is provided, the perceived cred¬ 
ibility of LCA study results may be enhanced [3]. However, as 
knowledge of the bases of the conversion methods are limited 
among practitioners and decision makers alike [3], the com¬ 
bined confidence parameter may unintentionally reduce un¬ 
derstanding of the uncertainty in the results. 

1.5 Methods for comparing system quality scores 

Quality assessment results are generally presented using box 
plots (Fig. 2) [19]. The cut-off probability for the boxes is 
chosen by the decision-maker [19], based on the allowable 
risk, i.e. 95% (95 out of every 100 possible cases is included). 
These plots may prove helpful, when there is minimal over¬ 
lap, as in cases A and B. However, when there is significant 
overlap, as in cases B and C, it is more difficult to determine 
which is the better option. 



Fig. 2: Box plots of notional system quality scores (the positions of the 1 st 
and 3rd percentiles is dictated by the desired confidence) 
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Fig. 3: Example cumulative normalised difference plot 

The usual method of resolving this difficulty is to perform 
the LCA on the difference between the cases (i.e. Case A 
minus Case B). If the generated quality box plot is entirely 
above zero, then Case A is definitely greater than Case B. 
When the value straddles zero, a value judgement must be 
made to decide between the options. Generally, the posi¬ 
tions of the mean (or perhaps the median) values are used as 
evidence of difference, and the degree of crossover reported, 
qualitatively or quantitatively. 

Another proposed method involves the generation of a nor¬ 
malised difference probability distribution [19], character¬ 
ised as the ratio (Result (Case A) - Result (CaseB ,)/Result (CaseA) . If 
this ratio is positive (i.e. 0.12) then Case A is said to be 
better than Case B by the magnitude of the ratio (i.e. 12%). 
Conversely, if the ratio is negative, then Case B is said to be 
better than Case A, again by the magnitude of the ratio. It is 
also proposed [19] that by preparing a graph of the cumula¬ 
tive distribution function of this ratio (Fig. 3), using a Monte- 
Carlo simulation, the probability of one case being signifi¬ 
cantly better than the other (say by a 10% margin) can be 
determined. For example, in Fig. 3 there is a 50% probabil¬ 
ity that Case B is better than Case A by at least 10%, while 
there is only a 20% chance that Case A is better than Case B 
by the same margin. This step is considered essential only if 
the alternatives cannot be separated through closer inspec¬ 
tion of the system. 

1.6 Current practice 

Data quality assessment is not current practice in LCA stud¬ 
ies [2,3,4]. A recent survey of LCA studies reported that 
only 4 out of 30 studies explicitly reported problems of un¬ 
certainty, and of these only one produced a quantitative 
analysis and two produced a qualitative analysis [2]. This 
situation however can be considered an improvement from 
an earlier study of data quality and databases [20], which 
reported no use of uncertainty analysis. During preparation 
of this case study on electricity generation, a similar trend 
was observed. Out of approximately 30 reports, only one 
report used a quantitative quality assessment technique (con¬ 
fidential report), one used a simplified quantitative method 
on a process level [21], another gave a qualitative discus¬ 
sion of uncertainty issues [22], and a few others gave indica¬ 
tive ranges based on the variability of a single data value, in 
this case electricity generation efficiency (i.e. [23,24]). 


This practice may reflect the already considerable time require¬ 
ments for many LCAs and a belief that detailed quality assess¬ 
ment does not provide sufficient benefit to decision makers to 
justify its expense. It has also been suggested that the current 
unsatisfactory state of data quality analysis and its lack of 
transparency [7] provides little incentive to include it in LCA 
studies. Thus, research into possible simplifications is impor¬ 
tant to reduce the need for detailed analysis of each datum [3], 
and the time required for data quality analysis [14]. Integra¬ 
tion of data quality methods into existing LCA software pro¬ 
grams could reduce time requirements [4]. 

2 Electricity Generation Case Study 

2.1 Introduction 

Electricity is one of the most important contributors to a 
nation's economy. In Australia, almost all electricity is pro¬ 
duced using fossil fuels (-90%), and of these black coal 
(55%) and brown coal (or lignite) (30%) are the dominant 
contributors [25]. The environmental impact of electricity 
from these two fuels is highly significant; for example, in 
1998 black and brown coal electricity generation contrib¬ 
uted 20% and 15% respectively of Australia's total green¬ 
house gas emissions [26]. 

2.2 Inventory data and impact assessment 

Inventory data was obtained from both confidential and 
public sources, and refers to the black coal electricity gen¬ 
eration industry in New South Wales and Queensland (Aus¬ 
tralia), and the brown coal electricity generation industry in 
Victoria (Australia). Ranges for some data were estimated 
using the 'rules of thumb' of Finnveden and Lindfors [11]. 

The impact assessment involved the use of impact factors 
provided by CML [7]. The impact categories chosen for the 
data quality study were climate change, acidification, and 
resource depletion, because they are generally regarded as 
the most significant effects of power generation (e.g. [21- 
24]). The molecular species of most significance for climate 
change are CO z , CH 4 and N z O from the combustion of fos¬ 
sil fuel, while for acidification they are NO x and SO x also 
from fuel combustion, and for resource depletion they are 
coal and other fuels. The impact factors used in the analysis 
also contain uncertainties, which will contribute to the un¬ 
certainty of the calculated impact scores. Important infor¬ 
mation such as temporal and spatial spread, dose-response, 
and thresholds, required for site specific studies, are ignored, 
which makes the conversion of material data to environ¬ 
mental effects a highly uncertain process [2]. This uncer¬ 
tainty was not considered in this analysis. 

2.3 System descriptions 

Both systems consist of mining, processing, and transport 
of coal, and electricity generation and transmission (Fig. 4). 
However, due to the quite different nature of the coals (Fig. 5) 
there are significant differences between black and brown 
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Required Materials 


Power Generation 
SYSTEM 


GENERATION 


Waste Treatment 


TRANSMISSION ■ -I- «* 
Electricity j 


Raw Materials from 
the environment 


ENVIRONMENT 


- Group of processes that do 

Waste Treatment die operation mentioned 
(Le. waste treatment) 

System Inputs: 

—Raw (untreated) materials (resources) 
SystEm Outputs: 

Emissions to the environment 


Distribution (Low Voltage) 
Network and End Use 


Boundary between the system 

- and the surrounding economy 

(System Boundary) 

► Treated material inputs 
* ► Emissions requiring treatment ~ 


Waste materials to 
the environment 

Boundary between the 
• • economy and the 

surro u nd ing e n viro nme nt 


Economic By-Products 


Fig. 4: System boundaries for the LCA case study: electricity generation from black or brown coals 


coal generation. In the black coal system some 25% of the coal flow through the initial stages. Both the black and brown 

coal is obtained from underground mines, and the remainder coal systems used pulverised fuel-steam turbine technology. 

(75%) is obtained from surface mines. When used for domes- The greater moisture content of brown coal reduces its elec- 

tic power production, the coal is only crushed and milled be- trical generation efficiency (electrical energy over coal energy 

fore consumption. Coal transport is by three methods: con- consumed) below that of an equivalent black coal plant (i.e. 

veyors (62% of coal, short range (5 km)); trucks (17%, mid 23-34% brown coals, 30 to 40% black coals). As a result, the 

range (10 km)); and trains (21 A, long range (210 km)). Coal environmental performance of a brown coal system is gener- 

may be stored at any stage of this system. In the brown coal al]y assumed worse than the equivalent black coal system, 

system, coal is continuously mined from surface mines, and 

the high moisture content ensures the coal is soft, removing For both systems, the transmission occurs over high voltage 

the hazards associated with the use of explosives, and the drill- wires to a substation, where the electricity's voltage is re- 

ing operations necessary for black coal mining. Coal process- duced for use in a distribution grid. Losses for both trans- 

ing involves only simple fan milling, without grinding or crush- mission grids are between 1 and 3 %. The functional unit is 

ing. Coal transport is exclusively via conveyors over short range one unit (MWh) of electricity delivered to this substation. 

(5 km). Coal storage is considered uneconomic. Final distribution and end use of electricity are excluded 

The technology used in producing electricity has a major in- from the systems because of the highly complex, and loca- 
fluence over the system output, as it defines the magnitude of tion specific nature of such systems. 



Fig. 5: Australian black and brown coal constituents 
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3 Results and Discussion 
3.1 Numerical uncertainty 

Monte-Carlo simulations (explained in Section 1.2) were 
completed for both black and brown coal cases. Varied in 
the simulations were all data necessary to produce the cli¬ 
mate change, acidification, and resource depletion indica¬ 
tors, except the impact factors (see Section 2.2). Each simu¬ 
lation involved 10000 calculations of each of these indicators, 
using XlSim® software. As many data point's uncertainty 
distributions were unknown, a number of common types 
were tested: triangular, uniform, normal and log normal (see 
Fig. 1). The maximum, minimum and mean values were used 
to define these distributions. The effect of this choice was 
found to be considerable (Table 4). The minimum mean value 
and standard deviation were obtained when using normal 
distributions, and the maximum standard deviations were 
attained when using uniform distributions. In the calcula¬ 
tion of the acidification potential of the brown coal system 
the use of triangular and uniform distributions was found 
to increase the mean impact value by over 300%. This was 
caused by the highly skewed uncertainty distribution of the 
NO x and SO x emission factors, whose mean and minimum 
values were similar and distant from their maximum val¬ 
ues. Their actual distribution was approximately log-nor¬ 
mal (see Fig. 1), and neither the uniform or the triangular 
distributions could successfully approximate a log-normal 
distribution. For a uniform distribution, its outputs will be 


spread evenly in the range, and so its mean must lie halfway 
between its extents. For a triangular distribution, its output 
distribution is limited such that its average must lie between 
29 and 71% of the distance between its extents. The fact 
that such problems can occur, limits the practicality of stand¬ 
ard, arbitrary distributions. 

When the distribution for each data point cannot be deter¬ 
mined (i.e. if insufficient data can be collected within the 
available time frame), then it is recommended that a skewed 
distribution, such as the triangular or log-normal distribu¬ 
tions, be used. They allow the average data point to be closer 
to either its minimum or maximum, and can be defined from 
a data point's average, minimum, and maximum values (see 
Fig. 1). In addition, the triangular distribution probably over¬ 
estimates the true uncertainty profile, which is likely to ap¬ 
proximate a skewed normal distribution, and thus provides 
a conservative estimate. 

Some additional simulations were completed to determine 
which data were the greatest contributors to the final uncer¬ 
tainty profile (Table 5). During these simulations, the uncer¬ 
tainty profile for a chosen data point was removed, and thus 
the chosen data point was then constant in all simulations. 
When the power station efficiency was held constant the 
standard deviations and therefore the variability of the re¬ 
sults were substantially reduced and the uncertainty ranges 
of the black and brown coal systems became similar. The 
greater uncertainty in the black coal data was due to the 


Table 4: Results of a numerical simulation using the Monte-Carlo technique (SO = standard deviation) 


System 

Probability 
Distribution Type 

Climate Change 
(kgco 2 -«i per MWh) 

Acidification 
(kSteo 2 iq. per MWh) 

Resource Depletion 
(kgskeq- per MWh) 



Mean 

SD 


SD 

Mean 

SD 

Black Coal 




. 



mmismgm 


Average 

1040 

- 

6.77 

- 

6.37 

~ 


Triangular 

1082 

309 

7.98 

2.90 

6.51 

1.92 


Uniform 

1080 

460 

9.32 

4.53 

4.55 

2.89 


Normal 

1038 

255 

6.88 

2.21 

6.40 

1.53 


Log Normal J 

1099 

334 

7.38 

2.83 

6.79 

2.16 

Brown Coal 


# -L'r.-O' 







Average 

1341 

- 

3.47 

- 


- 


Triangular 

1319 

102 

12.0 

6.1 

11.1 

0.6 


Uniform 

1386 

138 

19.3 

8.2 

11.3 

0.9 


Normal 

1338 

76 

3.34 

4.67 

10.9 

0.5 


Log Normal 

1346 

76 

4.92 

3.38 

10.9 

0.5 


Table 5: Simulations indicating the influence of individual data uncertainty on indicator uncertainty (Triangular distribution, SD = standard deviation), A 
from Table 4 


I .BH 

Climate Change (kgco 2 -eq. per MWh) 

Black Coal 

Brown Coal 

Mean 

SD 

Mean 

SD 

Triangular, No Variable Fix A 

1082 

309 

1319 

102 

Power Station Efficiency 

1084 

65 

1368 

59 

Coal Lost in Transit 

1111 

277 

1382 

101 

Coal C0 2 Emission Factor 

1055 

304 

1363 

80 

Coal-Bed Methane 

1085 

298 

1319 

102 
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wide range of electricity generation efficiencies encountered 
within the study scope, Australia. This variation is due to 
differences in age and design of electricity generation plant. 
Also an individual power station's efficiency may also fluc¬ 
tuate during operation, due to variances in fuel quality, ash 
deposition on boiler tubes, operation during startup and 
shutdown, and by capacity utilisation effects. In the brown 
coal case, the variation due to uncertainty in the emission 
factor of C0 2 from coal combustion was also a significant 
contributor, as the moisture content, and thus the amount 
of carbon per kg of raw coal, varies significantly between 
the three coal sources. Other data uncertainties, such as the 
amount of coal lost as dust in mining and transportation, 
and coal-bed methane emission factor, were found to be 
minor contributors to the uncertainty of the results. 

3.2 Qualitative uncertainty 

Qualitative uncertainty measures for the black and brown 
coal systems were obtained using a single set of indicators 
(see Table 1) to ensure a fair comparison. It may be signifi¬ 
cant that these indicators are all data level indicators, while 
both methods were developed to include process and sys¬ 
tem level indicators in their intended indicator sets. 

Qualitative uncertainties calculated using the method of 
Wrisberg (see Section 1.3) are shown in Table 6. They have 
been aggregated for each system section independently as 
well as for the total system to highlight the areas in which 
quality is poorest (i.e. has highest score). It was observed 
that reliability and completeness were the major contribu¬ 
tors to qualitative error for both black and brown coal cases. 
This was due to the use of some data that was unverified or 
based on assumptions (i.e. poor reliability), and the lack of 
data for some facilities within the desired region (Australia). 
The '%' column was added to show the relative contribu¬ 
tions of each section to the total indicator. Thus, as the trans¬ 
mission data indicators contribute less than 10% to the to¬ 
tals, increases in the quality of the transmission data will 
not substantially decrease the totals. Mining and transport 
contributed a higher than expected proportion to the totals, 


because in this method each important data flow is given 
equal weighting. Thus, even though the generation stage 
dominated the impact results (for climate change and acidi¬ 
fication), it had a similar number of streams contributing to 
the scores as the mining and transport stages, and so did not 
dominate the Wrisberg totals. Therefore, the final score was 
indicative of the quality of the inventory collection proce¬ 
dure (i.e. how good relative to theoretical perfection). How¬ 
ever, this method fails to establish whether quality is good 
in data that counts towards the impacts. As such, this meth¬ 
od's results cannot be used in discussions of the quality of 
the environmental impact results. 

Total qualitative uncertainties calculated using the method 
of Rousseaux et al. (see Section 1.3) are shown in Table 7. 
When a target quality score of 1 or 2 was set, the acceptabil¬ 
ity (i.e. the percentage of inventory scores equal to the tar¬ 
get score or less) for each indicator was lower for black coal 
than brown coal. This indicates that the data quality of the 
brown coal system was better than for the black coal sys¬ 
tem. For the reliability indicator, the use of industry data for 
much of the brown coal system, as opposed to the majority 
use of published reports for black coal system data, led to 
better scores. Also significant was the smaller number of 
existing brown coal systems (3 c.f. black coal >10), allow¬ 
ing more complete representation of the average brown coal 
system, and thus a better completeness score. Comparing 
variability scores, it was noticeable that both systems had 
high temporal scores, indicating a wide spread of indicator 
values. In the inventory of both systems most sources were 
recent (within three years of the year of study), which means 
an indicator score of 1 or 2 (see Table 1). However; a number 
of very detailed older sources (10 years prior to the year of 
study) were also used, giving indicator scores of 5. This was 
also true for the brown coal system reliability score, where 
a small number of estimates had to be made (indicator score 
5), while most data was verified and based on plant meas¬ 
urements (indicator score 1). This method thus shows more 
promise in locating and identifying the cause of deficiencies 
in data quality. However, like the method of Wrisberg, this 
method does not weight data quality based on the data's 


Table 6: Qualitative uncertainty scores calculated using the Wrisberg method 


Section 

:-a; . . 




Score 

■ 



wumm 

Reliability 

Completeness 

Temporal 

Geographical 

Technological 

Score 

% 

Score 

% 

y Score 

% J 

Score 

% 


% 

Black Coal 

ri; 







"MffSM §J 

''T ?>%- J 


Mining 

2.87 

35 

3.60 

40 

1.20 

29 

1.60 

39 

1.67 

42 

Transport 

3.57 

20 

4.14 

21 

2.14 

24 

1.71 

20 

1.29 

15 

Generation 

2.47 

38 

2.26 

32 

1.37 

41 

1.21 

38 

1.26 

40 

Transmission 

4.00 

7 

5.00 

7 

2.00 

6 

1.00 

3 

1.00 

3 

Total 

2.86 

100 

3.16 

100 

1.47 

100 

1.42 

100 

1.40 

100 

Brown Coal 





.. ' ' 


■ -as 


Mining 

2.82 

32 

2.64 

26 

1.00 

19 

1.18 

25 

1.36 

31 

Transport 

3.83 

23 

4.00 

21 

3.00 

32 

2.00 

23 

1.00 

12 

Generation 

1.64 

37 

2.23 

44 

1.09 

42 

1.14 

48 

1.18 

53 

Transmission 

4.00 

8 

5.00 

9 

2.00 

7 

1.00 

4 

1.00 

4 

Total 

2.39 

100 

2.73 

100 

1.39 

100 

1.27 

100 

1.20 

100 
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Table 7: Qualitative uncertainty scores calculated using the Rousseaux et al. method 


Parameter 


Acceptability at Objective Quality Score 


Variability 

.j .1:. ' ' 


3 

4 

2 

Black Coal 

r : '''''' ■ - 

0msm 5 :■ >')■ a:! 




Reliability 

0.0 

51.2 

76.7 

86.0 

40.9 

Completeness 

0.0 

48.8 

62.8 

72.1 

54.1 

Temporal 

79.1 

86.0 

93.0 

95.3 

75.9 

Geographical 

74.4 

86.0 

97.7 

100 

44.4 

Technological 

81.4 

81.4 

97.7 

100 

51.7 

Brown Coal 



■ ■: Hl 

:: 

, : i 

Reliability 

36.6 

61.0 

73.2 

90.2 

81.3 

Completeness 

2.4 

68.3 

78.0 

78.0 

58.6 

Temporal 

85.4 

90.2 

92.7 

92.7 

85.9 

Geographical 

80.5 

92.7 

100 

100 

27.7 

Technological 

90.2 

90.2 

100 

100 

30.2 


contribution to environmental impacts 4 , and thus also can¬ 
not be used to discuss the quality of impact results. 

3.3 Combined uncertainty 

Combined uncertainty measures for the black and brown 
coal systems were obtained using the methods of Kennedy 
et al. and Meier, using the indicators shown in Table 1. Pre¬ 
sented in Table 8 are those values obtained for climate 
change, along with the numerical simulation results reported 
in Table 4. For the black coal system, the use of the Kennedy 
et al. method modified the mean value by a small amount (± 
5%), while increasing the standard deviation (and hence 


6 The method of Lindeijer et al. 1 is reported to do this, however it is only 
available in Dutch. 


uncertainty) by 40-90%. The use of the Meier method had 
little effect on either the mean or the standard deviation. 
For the brown coal system, both methods left the mean un¬ 
changed, while increasing the standard deviation dramati¬ 
cally. In fact, the standard deviations of the Kennedy et al. 
brown coal systems were very similar to those of the black 
coal system, although still lower as a percentage of the mean 
values. Therefore, it was clear that the brown coal system 
had lower uncertainty than the black coal system. The two 
methods produce very different results, and there was no 
evidence that either approach produced results that were 
more accurate, or more representative of qualitative uncer¬ 
tainty, than the other. More importantly, it could not be dem¬ 
onstrated that either method produced a measure of uncer¬ 
tainty that was more relevant than that of the numerical 
uncertainty method alone. 


Table 8: Variation from numerical analysis values using the methods of Kennedy at al. and Meier (SD = Standard Deviation) 


Distribution 

Method 

: : 

Climate Change (kgco 2 ^q. per MWh) 

Black Coal 

Brown Coal 

Mean 

SD j 

Mean 

SD 

Triangle 

Numerical 

1082 

309 

1319 

102 


Kennedy et al. 

1062 

545 

1355 

558 


Meier 

1059 

331 

1367 

158 

Uniform 

Numerical 

1080 

460 

1386 

138 


Kennedy et al. 

1087 

663 

1423 

550 


Meier 

1108 

495 

1393 

230 

Normal 

Numerical 

1038 

255 

1338 

76 


Kennedy et al. 

1027 

478 

1354 

551 


Meier 

1021 

259 

1342 

135 

Lognormal 

Numerical 

1099 

334 

1346 

76 


Kennedy et al. 

1102 

572 

1331 

524 


Meier 

1074 

332 

1353 

139 
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Table 9: Differential numerical uncertainty ranges (brown coal - black coal, per MWh) 


System 

Probability 
Distribution Type 

Climate Change 
(kgco 2 -eq.) 

Acidification 

(kgsOj-eq.) 

■ . 1 

Resource Depletion 
(kgsb^q.) 

Mean 

SD 


SD 

Mean 

SD 

Black Coal - Brown Coal 

Average 

301 

- 

-3.30 

- 

4.54 

- 


Triangular 

237 

321 

4.02 

6.60 

4.55 

2.05 


Uniform 

306 

484 

10.0 

9.2 

6.77 

2.8 


Normal 

300 

267 

-3.53 

5.05 

4.52 

1.62 


Log Normal 

247 

341 

-2.46 

4.57 

4.15 

2.21 



Normalised Difference (Brown Coal-Black Coal)/Brown Coal (%) 

Fig. 6: Cumulative distribution function of the normalised difference 

3.4 Comparisons 

The differential case (brown coal system minus black coal 
system) was also calculated (Table 9) to determine which 
system had least environmental impact. While the average 
values indicated that the black coal system had lower cli¬ 
mate change and resource depletion impacts and higher acidi¬ 
fication impact, the numerical ranges overlapped indicating 
that these outcomes were not certain. Therefore, clear con¬ 
clusions could not be derived from this method. 

A normalised difference probability distribution plot was 
produced for the triangular climate change impact, calcu¬ 
lated as (brown coal-black coal)/brown coal (Fig. 6). This 
figure showed that there was a 12% chance that the brown 
coal system was at least 10% better than the black coal sys¬ 
tem, while there was a (100-41) or 59% chance that the 
black coal system was 10% better than the brown coal sys¬ 
tem. Thus, it could be concluded that the black coal system 
was more likely to produce a better result, as predicted by 
the averages. Thus, a clearer picture of the differences be¬ 
tween the systems and the potential for error was obtained. 

4 Conclusions 

Data quality assessment is important for the development 
and acceptance of LCA as a decision-making tool. Yet, data 
quality assessment is rarely performed, and there is no con¬ 
sensus on the methods to use. Methods for data quality as¬ 
sessment have recently undergone considerable development. 
However, increased demands on resources required for their 
implementation and doubts about resulting benefits have 


impeded adoption of any of the methods. Further, a defined 
means of reporting conclusions from uncertainty assessments 
in LCA, essential for comparing results from different stud¬ 
ies, remains to be elucidated. 

Six different methods of assessing quantitative and qualita¬ 
tive uncertainty have been used in a case study of electric 
power generation from Australian black coal and brown 
coal. Each method indicated higher uncertainty in the black 
coal system results than in the brown coal system results. 
The qualitative methods enabled the source of this differ¬ 
ence to be traced to the greater usage of data derived from 
plant measurements for the brown coal case. 

The numerical methods enabled a single uncertainty range 
to be reported for each environmental impact result, allow¬ 
ing the LCA results to be presented using standard quality 
assessment tools (i.e. box plots). The qualitative methods 
also presented significant data quality information, and al¬ 
lowed the identification of both the source and type of defi¬ 
ciencies in data quality. The methods which combine quali¬ 
tative uncertainty with quantitative uncertainty to provide 
quantitative outputs were not recommended. They were re¬ 
liant on subjective (expert) reasoning, provided no greater 
information than the quantitative methods, and presented 
far less information than if the qualitative and quantitative 
methods were presented separately. 

The'impacts calculated using average values of input data 
showed that the brown coal system had greater climate 
change and resource depletion impact scores, but a lower 
acidification impact score than the black coal system. How¬ 
ever, as the uncertainty ranges overlapped for each impact, 
it was shown that it is possible under certain circumstances 
for these relativities to be reversed. For example, if a black 
coal system operates at low efficiency, its climate change 
and resource depletion impacts can be greater than for a 
brown coal system, producing an equivalent amount of elec¬ 
tricity, but operating at high efficiency. The normalised dif¬ 
ference method was able to show, in an objective manner, 
that for the climate change impact, the black coal system 
has a greater probability of a lower impact result than the 
brown coal system. 

Therefore, it is recommended that impact results should be 
expressed using the results of a quantitative assessment, us¬ 
ing a skewed probability distribution (e.g. triangular or log 
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normal), supported by the qualitative assessment of Wrisberg. 
When the uncertainty ranges of impact results overlap, the 
normalised difference method should be used to rank the 
different options. 

As the data quality assessment methods are still underdevel¬ 
oped, a quality analysis is recommended only where the rela¬ 
tive merits of options are debatable and the magnitudes of long¬ 
term environmental and economic impacts are considerable. 
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